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Abstract 

The value of a zero-sum differential games is known to exist, under Isaacs' con- 
dition, as the unique viscosity solution of a Hamilton-Jacobi-Bellman equation. In 
this note we provide a self-contained proof based on the construction of e-optimal 
strategies, which is inspired by the "extremal aiming" method from j3]. 

1 Comparison of trajectories 

Let U and V be compact subsets of some euclidean space, let || • || be the euclidean 
norm in M", and let / : [0, 1] x x [/ x "5/ ^ M". Let H = {to < < • • • < be 
a set of times in [0, 1], and let ||n|| := maxi<m<N tm — im-i- For any Z C M", let 
D{x,Z) := mfz(zz \\x — z\\ be the usual distance to the set Z. 
Assumption 1: 

a. / is uniformly bounded, i.e. ||/|| := sup^^ x, u, ?;)|| < -|-oo. 

b. There exists c > such that: \f{u,v) eU x V, Vs,t € [0, 1], Vx^y G M", 

\\f{t,x,u,v) - f{s,y,u,v)\\ < c{\t -s\ + \\x - y\\). 

The local game: For any e [0,1] x and any ^ G M", consider the 

one-shot game with actions sets U and V and payoff function: 

{u,v) ^ {^,f{t,x,u,v)). 

Let this game be denoted by T(t,x,(^), and let H~{t,x,£,) and H~^{t,x,£,) be its 
maxmin and minmax respectively: 

H~{t,x,S,) := maxmin(^, /(t, X, M, ?;)), 

H+{t,x,0 ■■= mirimax(^,/(t,x,n,7;)). 

*The authors are particularly indebted with Pierre Cardaliaguet, Marc Quincampoix and Sylvain 
Sorin for their careful reading and comments on earlier drafts. This work was partially supported by 
the Commission of the European Communities under the 7th Framework Programme Marie Curie Initial 
Training Network (FP7-PEOPLE-2010-ITN), project SADCO, contract number 264735. 
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These functions satisfy H < H'^ . If the equahty H'^{t,x,S^) = H {t,x,(,) holds, 

the game r(t, x,^) has a value, denoted by H{t,x,S^). 

Assumption 2: T{t,x,C) has a value for aU {t,x,C) G [0, 1] x M" x M". 

We suppose that the Assumptions 1 and 2 hold in the rest of the paper. 

1.1 A key Lemma 

Introduce the sets of controls: 

U = {u : [0, 1] U, measurable}, V = {v : [0, 1] V, measurable}. 

Elements of U and V will be identified with constant controls. For any (to, zq, u, v) € 
[0,1] X X U X V denote by z[to, zq, u, v] the solution of: 

±{t) = f{t, Z(t), U(t), V(t)), z(to) = ZQ. 

Let (u, v) € Z// X V be a pair of controls, to € [0, 1] an initial time, {xo,wq) G (M")^ a 
pair of initial positions, and (u* ,v*) a couple of optimal actions in r(to, xq, xq — wq). 
Note any pair {u,v) U xV is optimal in this local game if xq = wq. Consider 
x(t) := x[tQ,XQ,u,v*]{t) and w(t) := w[tQ,W(),u*,v]{t). The following lemma is 
inspired by Lemma 2.3.1 in [3]. The existence of the value in the local games will 
be used to bound the distance between these two trajectories. 

Lemma 1.1. There exists real numbers A,B>0 such that for all t € [to, 1].' 

||x(t) - w(t)||2 < (1 + (t - to)A)\\xo - wof + B{t - tof. 
Proof. Let do '•= \\xo — wq\\ and d(t) := ||x(t) — w(t)||. Then: 

d\t) = \\[xo-wo)+ ff{sMs),n{s),v*)-f{sMs),n*Ms))dsf-(.l-l) 

J to 

The boundedness of / implies that: 



/(s,x(s),u(s),v*) - f{s,w{s),u*,v{s))ds 



to 



<4||/f(t-to)2. (L2) 



Claim: For all s € [to, 1], and for all {u,v) e U xV: 

{xq - Wq, f{s,x{s),u,v*) - f {s , w{s) , u* , v)ds) < 2C{s)do + cdl, (L3) 
where C{s) := c(l + ||/||)(s - to). 

Let us prove this claim. Assumption 1 implies ||x(s) — xo|| < (s — to)||/||, and then: 

\\f{sMs),n,v*) - f{to,xo,u,v*)\\ < c{{s - to) + - to)) = C{s). 

From Cauchy-Schwartz inequality and the optimality of v* one gets: 

{xo-wo,f{s,Jii{s),u,v*)) < {xo-wo,f{to,xo,u,v*)) + C{s)do, (1.4) 

< H~^{to,Xo,Xo - Wq) + C{s)do. (1.5) 
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Similarly, Assumption 1 implies ||w(s) — xo|| < do + (s ~ ^o)||/||) and then: 

\\f{s,w{s),u*,v) - f{to,xo,u*,v)\\ < C{s) + cdo. 

Using Cauchy-Schwartz inequality, and the optimality of u*: 

{xq - Wo, f{s, w{s),u*,v)) > (xo - Wo, /(to, xo, u*,v)) - (C(s) + cdo)do, (1.6) 

> H~{to,xo,xo - Wo) - C{s)do - cdl- (1.7) 

The claim follows: substract the inequalities (II. Sp anc (II. 7p and use Assumption 2 
to cancel {H'^ — H~){to, xo, xo — wo)- 

In particular, it holds for {u,v) = (u(s), v(s)). Note that J^* 2C{s)ds = {t — to)C{t). 
Thus, integrating (|1.3p over [to,t\ yields: 

/ {xo - wo,f{s,y.{s),\y{s),v*) - f{s,w{s),u*,v{s))ds) < {t - to){C{t)do + cdl). 

(1.8) 

Using the estimates (11.2^ and (ll.Sp in (ll.ip we obtain: 

d\t) <dl + 4\\ff{t - tof + 2{t - to)C{t)do + 2c{t - to)dl 

Finally, use the relations do < 1 + d^, C{t) < c(l + ||/||) and {t - to)C{t) = c(l + 
\\f\\){t-tof. The result follows with A = 3c+2||/|| and B = 4||/||2+2c(l + 1|/||). □ 

1.2 Consequences 

In this section, we give three direct consequences of Lemma [1.1 1 In Section [1.2.11 we 
use the set of times 11 to construct two trajectories on [to,i7v] inductively. Applying 
the lemma to the intervals [tm,tm+i], from m = to — 1, we obtain a bound 
for the distance between the two at time tjy. In particular, the distance vanishes as 
||n|| and ||xo — Wo\\ tend to 0. In Section [1.2.21 we replace the distance between two 
trajectories by the distance between a trajectory and a set. Finally, we combine the 
two aspects in Section [l. 2. 3j the result obtained therein is used in Section [2] to prove 
the existence of the value in zero-sum differential games with terminal payoff. 

1.2.1 Induction 

Let (u, v) € X V be a pair of controls. Define the trajectories x and w on [to, tAr] 
inductively: let x(to) = xq and w(to) = wo and suppose that x(t) and w(t) are 
defined on [to, tm] for some m = , . . . , A^ — 1; let {u^, v^) U x V he a couple of 
optimal actions in the local game r(tm, x(tm), x(tm) — w(tm)); for all t € [tm,tm+i], 
put x(t) := x[tm,x(tm),u,t;;;,](t) and w(t) := w[tm,w{tm),ul„v]{t). 

Corollary 1.1. |[x(t7v) - w(tAr) f < e^(||xo - wof + -B||n||). 

Proof. For any < m < A^, put dm '■= \\^{tm) — w(tm)||. By Lemma [LT] one has: 

+ B{tm — tm-l) ■ 

By induction, one obtains: 

d%<exp{A'y tm-tm~l)idl + By (tm-tm.~lf). 
^ — 'm.=l ^ — ^m=l 

The result follows using that ^.^^j^ t^— tm-l ^ 1 and ^^^^^(tm— tm-i)^ ^ l|n||- □ 
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1.2.2 Distance to a set 



Let W C [to,l] X M" be a set satisfying the following properties: 

• PI: For any t G [to, 1], W(t) := {x € | {t,x) G W} is closed and nonempty. 

• P2: For any {t,x) S W and any t' e [t, 1]: 

sup inf D{yL[t,x,u,v]{t'),W{t')) = 0. 
ugc/ vgv 

The first property ensures that the projection on W(t) is well defined for all t G 
[0, 1]. Equivalent formulations of the second property were introduced by Aubin [2], 
although the formulation P2 is inspired by the notion of stable bridge in p]. 

Let xq € M", let wq E argminy^;^^^^-) ||xo — wo|| be some closest point to xq in yV(to) 
and let v* be optimal in the local game T{tQ,XQ,XQ — wq). 

Corollary 1.2. For all t G [to, 1] and all u G hi: 

D\^[to, xo,u, v*]it), Wit)) < (1 + (t - to)^)^'(^o, >V(to)) + B{t - to)\ 

Proof. Let u G W be fixed. Let u* be optimal in T{to,X(),XQ — wq). By P2, for all 
e > there exists Vg G V such that the point MVeit) := x[tQ,Wo,u* ,V£]{t) satisfies 
D(ws{t),W{t)) < e. We use the following abbreviation: Xu(t) := x[to, xq, u, t;*](t). 
The triangular inequality gives D(xu(t), W(t)) < ||xu(t) — W£(t)|| + e. Taking the 
limit, as e ^ 0, one has that: 

D2(x,(t),>V(t)) < lim J|xu(t) - w,(t)f . 

£—5-0 

By Lemma [Til ||xu(t) - We(t)|p < (1 + (t - to)^)||xo - wof + B{t - t^f for all 
e > 0. The result follows because ||xo — Wo|| = -D(xo, VV(to)) by definition. □ 



1.2.3 A key Corollary 

Let Xq G W(to). For any u G iY, define a trajectory Xu on [to,tAr] inductively: let 
Xu(io) = a-iid suppose that Xu is defined on [to, ^m] for some m = 0, . . . , A^— 1. Let 
Wm G argmin^gy^;^^^^^) ||xu(tm) —1(^11 be a closest point to Xu(tm,) in VV(tm), and let v*^ 
be optimal in the local game r(tm, Xu(tm), Xu(tm) — ^^m)0 For all t G [tm,tm+i], put 
Xu(i) := x[tm, Xu(tm), u, ?;j^](t). Define a control /3(u) G V by setting /3(u) = v*^ on 
[im,im+i] for all < m < A^. Clearly, Xu(t) = x[to, xo, u, /3(u)](t), for all t G [to,t7v]. 

Note that the action -y*^ used in the interval [tm,,tm,+i] depends only on the 
current position Xu(tm) and on the set VV(tm), and that the former is a deterministic 
function oivl,.. .,v^_ _i and of the restriction of u to the interval [to,tm]- 

Consider /3 as a mapping from U to V. Then, for any ui,U2 & U such that 
ui = U2 on [to,tm] for some < m < A^, /3(ui) = /3(u2) on [to,tm+i]- In this sense, 
/3 : — )• V is nonanticipative. 

Putting Corollaries 11.11 and 11.21 together, and using that D(xo,W(to)) = 0, we 
obtain the following result. 

^We implicitly use two selection rules tti ~ 7ri(>V) and 712 defined as follows: tti : [0, 1] x M" — > R" 
assigns to each {t,x) a closest point to x in W(t); tt2 : [0,1] x R" x M" V assigns to each (i, a;,^) an 
optimal action in the local game r(i, x, ^). 
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Corollary 1.3. ForanyueU, L'2(x[to, 2:0, u, /3(u)](tAr), W(tAr)) < e^S||n||. 

This result can be interpreted as follows: suppose the state belongs to W at 
time to; then for any control u G (say, of player 1), there exists a piece-wise, 
nonanticipative reply of player 2 such that the distance to W at some terminal time 
vanishes as ||n|| tends to 0. In terms of strategies, which will be defined in the next 
Section, the result implies that player 2 has a strategy which keeps the state very 
close to the set W. This property is used in Section [2] to prove the existence of the 
value in zero-sum differential game. The epigraph of the lower value function will 
play the role of the set W. 

2 Differential Games 

For any (to,Xo) € [0, 1] x M", consider now the zero-sum differential game played in 
[to, 1] with the following dynamics: 

x{to) = xq, and x(t) = /(t, x(t), u(t), v(t)), a.e. on [to, !]• 

Definition 2.1. A strategy for player 2 is a map P : U ^ V such that, for some 
finite partition sq < si < • • • < sat of [to,l], for all ui, U2 (zU and < m < N: 

ui = U2 a.e. on[so,Sm] =^ /3(ui) = /3(u2) a.e. on[so,Sm+i]- 

These strategies are called nonanticipative strategies with delay (NAD) in [1], in 
contrast to the classical nonanticipative strategies. The strategies for player 1 are 
defined in a dual manner. Let A (resp. B) the set of strategies for Player 1 (resp. 2). 
For any pair of strategies (a, /3) (z A x B, there exists a unique pair (u, v) G x V 
such that a(v) = u, and /3(u) = v (see [1]). This fact is crucial for it allows to 
define x[tQ,xo,a, f3] := x[tQ,XQ,u,v] in a unique manner. 

The payoff in a differential game has generally two components: a running payoff 
and a terminal payoff, represented by the functions 7 : [0, 1] x x U x V ^ M. and 
g : M" M respectively. However, the classical transformation of a Bolza problem 
into a Mayer problem, which gets rid of the running payoff, can also be applied 
here: enlarge the state space from R" to M""''^, where the last coordinate represents 
the cumulated payoff; define an auxiliary terminal payoff function g : M"+^ ^ M as 
g{x, y) = g{x) + y; we thus obtain an equivalent differential game with no running 
payoff. W.l.o.g. we assume from now on that 7 = 0. Assumption 3: g is Lipschitz 
continuous. 

We suppose that the Assumptions 1, 2 and 3 hold in the rest of the paper. The 
differential game with initial time to, initial state xq and terminal payoff g is denoted 
by Q{to,xo). 

Introduce the lower and upper value functions: 

V~{to,xo) := sup inf 5r(x[to,2;o,a,/3](l)), 
V^ito^xo) := inf sup g{x[to, xq, a, /3]{l)). 

The inequality V~ < holds everywhere. If V~{tQ,XQ) = V~^{tQ,XQ), the game 
Q{tQ,X()) has a value. Notice that its lower and upper Hamiltonian of are precisely 
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the maxmin and the minmax of the local games defined in Section [TJ Consequently, 
Assumption 2 is precisely Isaacs' condition. 

The lower value function satisfies the following super-dynamic programming princi- 
ple (see [1]). 

For ah {t,x) G [to, 1] x M" and all t' G [to, 1]: 

V~{t,x) > supinf y~(t',x[to,xo,a,/3](t')- (2-1) 
A proof of a slightly weaker version of (j2.ip can be found in the Appendix. 

2.1 Existence and characterization of the value 

Let (f) : [tQ,l] X — > M be a real function satisfying the following properties: 
(i) X i-T- (j){t,x) is lower semicontinuous, for all t € [toi 1]; 
{ii) For ah {t,x) G [to, 1] x R" and t' G [t, 1]: 

(j){t,x) > sup inf (j)(t' ,x.[t, x,u,v]{t')y, 
{in) (j){l,x) > g{x), for all x G M". 

Definition 2.2. For any £ G M, define the £-level set of (f) by: 

Wf = {{t, x) G [to, 1] X M" I (Pit, x) < i}. 

Lemma 2.1. For any (. > (j){tQ,XQ), the (.-level set of (j) satisfies PI and P2. 

Proof, xq G W^(to) so that W^(to) is nonempty. By (i), VV^(t) is a closed set for 
all t G [0, 1]. The property {ii) implies that for any t G [to, 1], u G C/ and n G N* 
there exists v„ G V such that: 

^ > (t>{to,xo) > (l){t,x[to,xo,u,Vn]{t)) - ^. (2.2) 

The boundedness of / implies that Xn '■= x[to, xq, it, v„](t) belongs to some compact 
set. Consider a subsequence {xn)n such that lim„_>.oo (^{t, Xn) = liminf„_>.oo (/>(t, 
and such that {xn)n converges to some x G M". Take the limit, as n — > oo, in (12. 2p . 
Using {i) again, we obtain: 

£ > (t){to,XQ) > (p{t,x). 

Consequently, x G Wf{t) 7^ and inf^gpj* d(x[to, xq, li, v„](t), w/(t)) = 0. The 
proof of these two properties still holds by replacing the initial data (i.e. (tg, Xo) and 
t G [to, 1]) by some (t,x) G Wf and t' G [t, 1]. Thus, satisfies PI and P2. 

□ 
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2.1.1 Extremal strategies in Q(to,xo) 

Let C [to,l] X R"^ be the (f){to,Xo)-leve\ set of (p. Let vri = 7ri(VV"?^) and 7r2 be 
two selection rules defined as follows: vri : [0, 1] x M" — )• M" assigns to each (t, x) a 
closest point to x in >V''^(i); tt2 : [0, 1] x x M" ^> assigns to each (t, x,^) an 
optimal action in the local game r(t,x,^). Finally, let: 

TT : [0, 1] X M" V, {t,x) ^ TT2{t,X,X- TTl{t,x)). 

Definition 2.3. An extremal strategy /3 = 11, vr) : U ^ V is defined inductively 
as follows: suppose that (3 is already defined in \tQ,tm\ for some < m < N , and let 

= x[to,Xo,U,/3](t„). Set /3{u) = 7r{tm,Xm) on [tm.,tm.+l]- 

These strategies are inspired by the extremal aiming method of Krasovskii and 
Subbotin (see Section 2.4 in [3]). 

Proposition 2.1. For some C > 0, and for any extremal strategy (3 = (3[(j)^Ii,Ti): 

5(x[to,xo,u,/3(u)](l)) <,^(to,xo)+C^M, VuGZ^. 

Proof. Recall that xj^ = x[to, xq, u, /3(u)](l). By Lemma ETT] W"'^ satsifies PI and 
P2. Thus, by Corollary O 

D'^{xN,W^{tN)) < e^S||n||. (2.3) 

Using [Hi) and the fact that ^at = 1 yields: 

W^itN) = {x G M"| </.(l,x) < </.(to,xo)} C {x G M"| g{x) < </.(to,Xo)}. 

Let wn G argmin^gy^;^^;^) II^at — w\\ be some closest point to xn in W^(l). Let K be 
the Lipschitz constant g (assumption 3). Then: 

g{x]y) < g{wN) + fi\\xN - wn\\, 

< ^ito,Xo)+Kd{xN,W^itN)). 

The result follows from (j2.3p . □ 

Proposition 12.11 applies to any function satisfying (z), (ii) and (Hi). Consequently, 
under Assumptions 1, 2 and 3: 

V^ito^xo) < inf{0(to,xo) I (f) : [to, 1] x ^ R satisfying (i), {ii), {Hi)}. (2.4) 

Consequently, the value exists if the lower value function V~ satisfies {i) — {Hi). 

Tiieorem 2.4. The differential game Q{tQ,XQ) has a value, characterized as: 

V(to,xo)= inf (/)(to,xo). 

satisfying 
{i),{ii),{iii) 

The strategies /3(V,n) are asymptotically optimal for player 2, as \\Il\\ 0. 
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Proof. By definition, x) = g{x), for all x € M", so that (iii) is satified. The 

property (ii) can be deduced directly from (12.1^ : Assumption 1 and 3 imply that 
the map x i— t- V~{t,x) is Lipschitz continuous for all t G [to, 1]; in particular, (i) is 
satisfied. These properties being classical, we have preferred to give the details in 
the Appendix. Finally, let /? = /3(V, IT, vr) be an extremal strategy. Proposition 12.11 
gives: 

V+{to,xo) < supg(x[to,xo,u,/3(u)](l)) < V~ {to,xo) + C^/M- 

The existence of the value is obtained by letting ||n|| tend to 0. Moreover, note that 
for any e > 0, /3 is e-optimal for sufficiently small ||n||. □ 



3 Appendix 



Note that the classical subdynamic programming principle (|2.ip implies (ii). Indeed, 
any u ^ U can be identified with a strategy that plays u on [to, 1] regardless of v. 
Then: 

sup inf x[to, xo, a, /3])(t') > sup inf V~ {t' ,ii.[tQ,XQ,u, j3{u)]){t') 

= sup inf y~(t', x[to, xo, v])(t'). 

The proofs of (j2.ip and (ii) are essentially the same. We provide here a proof of the 
latter because it is this version that we have used in the proof of Theorem 12.41 

Claim 3.1. satisfies (i) and (ii). 

Proof, (i): Using Assumption 1 and Gronwall's lemma one obtains that, for all 
t£ [to, I], (u,v) eUxV, and x,2/GM": 

||x[to,x,u,v](t)) -x[to,y,u,v](t))| < e^(*-*°)||a;-y||. 

Thus, by Assumption 3, for all (u, v) € x V, and for all x,y ^ M": 

|5(x[to,x,u,v](l)) -5(x[to,y,u,v](l))| < Ke^(i-*o)|Ix-y||. 

Consequently, the map x V~{t,x) is Ke'^-Lipschitz continuous for all t € [to, !]• 
(ii): Let (t,x) € [to, 1] x R", t' G [t, 1] and e > be fixed. The Lipschitz continuity 
of z 1-^ V~{t' , z) implies the existence of some 5 > Q such that any e-optimal action 
in Q{t',x') is 2e-optimal in Q{t',z), for all z E B{x',5), which is the euclidean ball 
of radius 5 centered in x' . By compactness, let B{x, \\f\\) be covered by some finite 
family {Ei)i^j of pairwise disjoint sets, each one included in a ball B(xi,S) for some 
{xi)i S (M")^. Let ai ^ A he an e-optimal strategy for player 1 in V~ {t' , Xi), and 
use the notation Xv(t') := x[x, t, n, v](t'). Then, by definition, for all v 

5(x[t',Xv(t'),a.,v](l))l{,^(t,)eE,} > V^-(t',x(t'))l{x,(t.)6£;j -2e. 
For each u (z U, define a strategy G ^ as follows: Vt G [to, 1], Vv G V, 



a«(v)(t) 



u iftG[t,t'), 

ai(v)(t) iftG[t',l], and x^{t') e Ei. 



Note that is a NAD strategy in G{t,x). Indeed, let si < • • • < sat be a common 
partition of [t', 1] for the strategies {ai)i - this is possible because the family is finite. 
Thus, Ou is defined with respect to t < < S2 <•••< s^r. Now, for all v € V: 

C/(x[x,t,a„, V](l)) = ^.^^5r(x[t',Xv(t'),ai, V](l))l|x^(i/)g£;j, 

= y-(t',xv(t'))-2e, 
Taking the infimum in V, and the supremum in U yields the desired result. 

□ 
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